Skip to content

feat: implement cheap clone via Arc<Config> for all LLM backends#103

Merged
graniet merged 2 commits intograniet:mainfrom
aniketfuryrocks:feat/cheap-clone-arc-config
Jan 7, 2026
Merged

feat: implement cheap clone via Arc<Config> for all LLM backends#103
graniet merged 2 commits intograniet:mainfrom
aniketfuryrocks:feat/cheap-clone-arc-config

Conversation

@aniketfuryrocks
Copy link
Copy Markdown
Contributor

@aniketfuryrocks aniketfuryrocks commented Jan 5, 2026

Summary

This PR wraps backend configurations in Arc to enable O(1) cloning instead of expensive deep copies. All backend structs now derive Clone.

  • Wrap config fields in Arc<*Config> for all backends
  • Make config structs and fields public for direct access
  • Make client field public on all backends
  • Add with_client() constructors for HTTP client injection/pooling
  • Add doc comments to all config struct fields

Backends Updated

Standalone backends:

  • Anthropic, Google, Ollama, DeepSeek, XAI, AzureOpenAI, Phind, ElevenLabs

OpenAI-compatible backends (via OpenAICompatibleProvider):

  • OpenAI, Groq, Mistral, Cohere, OpenRouter, HuggingFace

Usage

// Clone is now cheap (Arc increment)
let provider2 = provider.clone();

// Direct field access
let api_key = &provider.config.api_key;
let client = &provider.client;

// HTTP client injection for connection pooling
let shared_client = reqwest::Client::new();
let provider = Anthropic::with_client(shared_client, "api-key", ...);

Testing

  • All 63 unit tests pass
  • All 12 doc tests pass
  • Clippy passes

Draft assisted with OpenCode

@aniketfuryrocks
feat: implement cheap clone via Arc<Config> for all LLM backends
2502402 
This change wraps backend configurations in Arc to enable O(1) cloning
instead of expensive deep copies. All backend structs now derive Clone.
Changes:
- Wrap config fields in Arc<*Config> for all backends
- Make config structs and fields public for direct access
- Make client field public on all backends
- Add with_client() constructors for HTTP client injection/pooling
- Add doc comments to all config struct fields
- Remove redundant accessor methods (use direct field access instead)
Backends updated:
- Anthropic, Google, Ollama, DeepSeek, XAI, AzureOpenAI, Phind, ElevenLabs
- OpenAICompatibleProvider (used by OpenAI, Groq, Mistral, Cohere,
  OpenRouter, HuggingFace)
Breaking changes:
- Previously private config fields are now accessed via .config.field
- client() getter removed, use .client directly
Draft assisted with OpenCode.
@aniketfuryrocks
add accessor methods to all backends for backward compatibility
00197aa
@aniketfuryrocks
Copy link
Copy Markdown
Contributor Author

aniketfuryrocks commented Jan 5, 2026 •
edited
Loading

@graniet quickly drafted this PR with opencode, let me know how it looks. I tried to make minimal changes. I think we need to refactor the backend directory to correctly couple related code in the future. This is a follow up pr with my comments on #100

@graniet @nazq

@aniketfuryrocks aniketfuryrocks marked this pull request as draft January 5, 2026 07:46
@aniketfuryrocks aniketfuryrocks marked this pull request as ready for review January 5, 2026 07:46
@graniet
Copy link
Copy Markdown
Owner

graniet commented Jan 5, 2026

Thank you @aniketfuryrocks for your work, I will review soon !

@graniet
Copy link
Copy Markdown
Owner

graniet commented Jan 7, 2026

LGTM !

@graniet graniet merged commit 0886d2d into graniet:main Jan 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@aniketfuryrocks @graniet